Who am I?

• https://pschmitt.net
• Academics
  • PhD at UCSB
  • Researcher (summers) at ICSI
  • Postdoc then research faculty at Princeton
  • Some other stops… USC, Stanford, UH
• Privacy startup co-founder & CEO, measurement startup COO
• Main focuses
  • Integrating privacy and security into systems that we all use
  • ML for network management
  • Connectivity / privacy in challenging environments

This class

• Too much to cover in one term: survey of important topics
• Course Canvas: https://canvas.calpoly.edu/courses/168889
• Syllabus: https://schmittpaul.github.io/CSC453W26/syllabus.pdf
• Waitlists…

Who are you?

• Name / Major / Year
• Where do you identify as your hometown?

How to succeed in this class

• Turn things in on time
• Attend class
  • … on time. Lectures will start at 10 after the hour
    
• Ask questions
  • I’m very flexible about how much we cover this semester
  • I would rather teach less and have everyone understand it
  • Our back-and-forth during class is the one of the few indicators I have of how much you are absorbing
• Talk to me if you are struggling

What are we doing here?

Why take this class

• You are required to do so in order to graduate
• Reality: this is how computers really work, and as a computer scientist or engineer you should know how computers really work
• Ubiquity: operating systems are everywhere and you are likely to eventually encounter them or their limitations
• Beauty: operating systems are examples of mature solutions to difficult design and engineering problems. Studying them will improve your ability to design and implement abstractions

Course progression

• Introduction to operating system abstractions and structures
• Abstracting and multiplexing:
  • CPU: interrupts, context, threads, processes, processor scheduling, thread synchronization, deadlocks
  • Memory: memory layout, address translation, paging and segmentation, address spaces, translation caching, page fault handling, page eviction, swapping
  • File systems and storage: disk scheduling, on-disk layout, files, buffer cache, crash and recovery
  • Security (time permitting)

Intro

• What is an operating system?
• What do they do?
• How did OSes evolve?
• How do OSes manage access to resources?

What is an operating system?

OS history - how did we get here?

Types of operating systems

• Desktop
• Time share/Mainframe
• Mobile
• Web
• Real-time
• Embedded
• Virtual Machines
• Q: How do the requirements differ?

Tl;dr

OSes continue to evolve as environments (i.e., constraints) change

Abstractions

• Abstractions simplify application design by:
  • hiding undesirable properties,
  • adding new capabilities, and
  • organizing information
• Abstractions provide an interface to application programmers that separates policy—what the interface commits to accomplishing—from mechanism—how the interface is implemented.

What are the abstractions?

• CPUs
  • Processes, threads
• Memory
  • Address space
• Disk
  • Files

Example OS abstraction: file systems

• What undesirable properties do file systems hide?
  • Disks are slow!
  • Chunks of storage are actually distributed all over the disk
  • Disk storage may fail!
• What new capabilities do files add?
  • Growth and shrinking
  • Organization into directories, searchability
• What information do files help organize?
  • Ownership and permissions
  • Access time, modification time, type, etc.

Abstraction tradeoffs - discussion

• Identify undesirable properties hidden by, new capabilities added, and info organization provided with these abstractions:
  • Process / threads
  • Address space

Abstraction pros / cons

• Advantages of OS providing abstractions?
  • Allow applications to reuse common facilities
  • Make different devices look the same
  • Provide higher-level or more useful functionality
• Challenges?
  • What are the correct abstractions?
  • How much should be exposed?

OS design requirements - what do we need?

• Convenience, abstraction of hardware resources for user programs
• Efficiency of usage of CPU, memory, etc.
• Isolation between multiple processes
• Reliability, the OS must not fail
• Other:
  • Security
  • Mobility

OS as a resource manager

• Another view: resource manager - shares resources “well”
• Advantages of the OS managing resources?
  • Protect applications from one another
  • Provide efficient access to resources (cost, time, energy)
  • Provide fair access to resources
• Challenges?
  • What are the correct mechanisms?
  • What are the correct policies?

Resources are managed via services

• Program Execution (loading, running, monitoring, terminating)
• Performance (optimizing resources under constraints)
• Correctness (overseeing critical operations, preventing interference)
• Fairness (access to and allocation of resources)
• Error detection & recovery (network partition & media failure)

Services (cont’d)

• Communication (inter-process, software-to-hardware, hardware-to-hardware, system-to-system, wide-area)
• I/O: reading & writing, support for various mediums, devices, performance, and protections
• Data Organization (naming), Services (search) & Protection (access control)
• Security (isolation, enforcement, services, authentication, accounting and logging, trust)
• User interfaces (command-line, GUIs, multiple users)

Each service has challenges and tensions

Example 1: We have limited RAM, and we want to run more programs that can be stored.

• How do we allocate space?
• Who stays?
• Who goes?
• What if we’re wrong?
• What if the system is under extremely heavy load?
• Is there a way to predict the future?

Each service has challenges and tensions

Example 2: We have two process (producer / consumer); how do they communicate?

• Message passing? Shared memory?
• How do they synchronize?
  • How to we prevent over-production? Over-consumption?
  • Context-switching?

Briefly: what happens at boot?

• BIOS / UEFI initializes devices and finds the next program to run
• Bootloader: simple code to initialize the system, load the kernel
  • Why do we need a bootloader?
    • Safe mode
• Kernel loads
  • First process: init (example: systemd), pid == 1
    • Parent of all processes
    • Init starts system daemons (services provided outside of the kernel)

Paradigms

• The OS offers a number of services. What should go in the kernel?
  • IPC
  • VFS
  • File system
  • Scheduler
  • Virtual Memory
  • Device drivers

Monolithic kernels

• Oldest, very common design (Linux, Windows 9x, BSDs)
• Single piece of code in memory
• Limited information hiding
  • One part of the kernel can directly access data and functions of other parts

Monolithic kernels (cont’d)

• Q: What happens when you need something new supported (new hardware device, new system call, new filesystem)?
• Modules (loadable kernel modules - LKMs) allow for flexibility, customization, support
  • Pros?
    • Memory savings
    • Flexibility
    • Minimal downtime
  • Cons?
    • (minor) fragmentation: the base kernel can be loaded contiguously in memory
    • Security (https://github.com/m0nad/Diamorphine DO NOT USE THIS, IT’S SIMPLY TO SHOW YOU)

Layered kernels

• Dijkstra created the THE OS in the 60s, introducing the concept Each inner layer is more privileged; required a trap to move down layers
• Hardware-enforcement possible
• Intel announced in May 2024 that the new architectures will only have rings 0 and 3

Microkernels

Monolithic vs Microkernel

Process isolation

Q: How can we ensure that a user process doesn’t harm others?

Dual-mode operation

• Dual-mode operation allows OS to protect itself and components
  • User mode and kernel mode
• Mode bit provided by hardware
  • Provides ability to distinguish when system is running user code or kernel code.
  • When a user is running → mode bit is “user”
  • When kernel code is executing → mode bit is “kernel”
• System call changes mode to kernel, return from call resets it to user
• Some instructions are only executable in kernel mode

System calls

• The OS offers a number of services. How do we (applications) interface with them?
  • We don’t want to deal with the details, just the abstraction
  • The OS has ultimate control over these operations
• System calls are the “language” of communication with the OS
• Standards
  • Win32 (MS)
  • POSIX (nearly all Unix-based systems)
  • Java API for the JVM

System calls (cont’d)

• Like a function call, we push arguments onto the stack, then we call into the library that provides the system call
• Each system call has a special number, placed into a register
• Executes a TRAP instruction (switch to kernel mode)
• A logical separation of memory space
• Kernel’s system call handler is invoked, once done (but may block) may be returned to the process

System calls (cont’d)

• Table defined in the kernel: https://github.com/torvalds/linux/blob/master/arch/x86/entry/syscalls/syscall_32.tbl
• You can run using the table values themselves using the syscall() wrapper
  • Q: why does syscall() exist?
  • If you’re interested… There are debates https://lwn.net/Articles/771441/

SLOsh

• This assignment should be a refresher of 357 topics
• Your shell will support:
  • basic command execution
  • built-in commands
  • signal handling
  • I/O redirection
  • piping

Requirements

• C99 POSIX
• Must compile / run on the unix* servers
• You must create a functional Makefile
• Submit a tar (.tgz) file in Canvas

Command execution

• Execute external commands
  • Use fork(), exec(), and waitpid() to manage process execution
• Support command-line arguments
• Handle both relative and absolute paths to executables
• Report command execution errors appropriately

Redirection / piping

• Use open(), dup2(), and close() for file redirection
• Use pipe() for creating pipes between processes

Signal handling

• Signals are async. They can happen at any time
• Certain functions are async-safe, others are not
  • printf() contains internal buffers and state; an interruption can lead to corrupted buffers

Handling SIGINT

• write() a newline or whatever to signify the signal was caught.
• If there is not a child running, write() a new prompt
• Why not just use display_prompt()?
  • If it’s using write() it could be fine. If it’s using printf() it isn’t

Parent vs child

• The child should immediately reset signal handling to the default

• Look up signal()’s man

  man 2 signal

Sigaction

• sigaction() is newer than signal()
• Offers more control and is more consistent across systems
• Allows specifying flags
  • SA_RESTART (man 7 signal)
    • Without it, syscalls return with EINTR error if they are interrupted by a signal
    • With it, most syscalls automatically restart
• Is the POSIX-compliant approach
• EINTR can still happen, it’s wise to check for it and handle it
  • When reading input from user